Detection of abnormal passenger behaviors on ships, using RGBD cameras

El objetivo de este trabajo fin de Máster (TFM) es el diseño, implementación, y evaluación de un sistema inteligente de videovigilancia, que permita la detección, seguimiento y conteo de personas, así como la detección de estampidas, para grandes embarcaciones. El sistema desarrollado debe ser portable, y funcionar en tiempo real. Para ello se ha realizado un estudio de las tecnologías disponibles en sistemas embebidos, para elegir las que mejor se adecúan al objetivo del TFM. Se ha desarrollado un sistema de detección de personas basado en una MobileNet-SSD, complementado con un banco de filtros de Kalman para el seguimiento. Además, se ha incorporado un detector de estampidas basado en el análisis de la entropía del flujo óptico. Todo ello se ha implementado y evaluado en un dispositivo embebido que incluye una unidad VPU. Los resultados obtenidos han permitido validar la propuesta.The aim of this Final Master Thesis (TFM) is the design, implementation and evaluation of an intelligent video surveillance system that allows the detection, monitoring and counting of people, as well as the detection of stampedes, for large ships. The developed system must be portable and work in real time. To this end, a study has been carried out of the technologies available in embedded systems, in order to choose those that best suit the objective of the TFM. A people detection system based on a MobileNetSSD has been developed, complemented by a Kalman filter bank for monitoring. In addition, a stampede detector based on optical flow entropy analysis has been incorporated. All this has been implemented and evaluated in an embedded device that includes a Vision Processing Unit (VPU) unit. The results obtained have allowed the validation of the proposal.Máster Universitario en Ingeniería de Telecomunicación (M125

Clasificación de accesorios a partir de información de profundidad

El objetivo de este trabajo de fin de grado (TFG) es la identificación robusta de complementos a partir de imágenes de profundidad (2.5D). Dichas imágenes serán adquiridas de una cámara Kinect II ubicada en una posición cenital. Los complementos evaluados en este caso son gorras y distintos tipos de sombreros (grandes, pequeños y medianos), que la solución propuesta debe ser capaz de identificar. La solución propuesta extrae un conjunto de descriptores por cada persona previamente detectada en la escena que, posteriormente, son clasificados utilizando la técnica PCA (Análisis de Componentes Principales), comparándolos con las distintas clases previamente entrenadas. El sistema desarrollado se ha evaluado realizando diferentes pruebas experimentales sobre secuencias de profundidad reales, obteniendo resultados satisfactorios. En concreto, se han obtenido tasas de acierto del 98% para el caso más sencillo (clasificación binaria) y superiores al 85% en los casos más complejos (cuatro o cinco clases).The aim of this final degree thesis is the robust identification of headgear accesories from depth images (2.5D) acquired using a Kinect II camera located in a zenithal position. The accesories evaluated in this work are caps and different types of hats (large, small and medium). The proposed solution must be able to identify complements of each class. The proposed solution extracts a set of descriptors for each person previously detected in the scene, which are then classified using the PCA (Principal Component Analysis) technique, comparing them with the different classes previously trained. The developed system has been evaluated by carrying out different experimental tests on real depth sequences, obtaining satisfactory results. Specifically, success rates of 98% have been obtained for the simplest case (binary classification) and higher than 85% in the most complex cases (four or five classes).Grado en Ingeniería Electrónica de Comunicacione

A new framework for deep learning video based Human Action Recognition on the edge

Nowadays, video surveillance systems are commonly found in most public and private spaces. These systems typically consist of a network of cameras that feed into a central node. However, the processing aspect is evolving towards distributed approaches, leveraging edge-computing. These distributed systems are capable of effectively addressing the detection of people or events at each individual node. Most of these systems, rely on the use of deep-learning and segmentation algorithms which enable them to achieve high performance, but usually with a significant computational cost, hindering real-time execution. This paper presents an approach for people detection and action recognition in the wild, optimized for running on the edge, and that is able to work in real-time, in an embedded platform. Human Action Recognition (HAR) is performed by using a Recurrent Neural Network (RNN), specifically a Long Short-Term Memory (LSTM). The input to the LSTM is an ad-hoc, lightweight feature vector obtained from the bounding box of each detected person in the video surveillance image. The resulting system is highly portable and easily scalable, providing a powerful tool for real-world video surveillance applications (in the wild and real-time action recognition). The proposal has been exhaustively evaluated and compared against other state-of-the-art (SOTA) proposals in five datasets, including four widely used (KTH, WEIZMAN, WVU, IXMAX) and a novel one (GBA) recorded in the wild, that includes several people performing different actions simultaneously. The obtained results validate the proposal, since it achieves SOTA accuracy within a much more complicated video surveillance real scenario, and using a lightweight embedded hardware.European CommissionAgencia Estatal de InvestigaciónUniversidad de Alcal

Smart Video Surveillance System Based on Edge Computing

New processing methods based on artificial intelligence (AI) and deep learning are replacing traditional computer vision algorithms. The more advanced systems can process huge amounts of data in large computing facilities. In contrast, this paper presents a smart video surveillance system executing AI algorithms in low power consumption embedded devices. The computer vision algorithm, typical for surveillance applications, aims to detect, count and track people’s movements in the area. This application requires a distributed smart camera system. The proposed AI application allows detecting people in the surveillance area using a MobileNet-SSD architecture. In addition, using a robust Kalman filter bank, the algorithm can keep track of people in the video also providing people counting information. The detection results are excellent considering the constraints imposed on the process. The selected architecture for the edge node is based on a UpSquared2 device that includes a vision processor unit (VPU) capable of accelerating the AI CNN inference. The results section provides information about the image processing time when multiple video cameras are connected to the same edge node, people detection precision and recall curves, and the energy consumption of the system. The discussion of results shows the usefulness of deploying this smart camera node throughout a distributed surveillance system